摘要 :
A number of structural coverage criteria have been proposed to measure the adequacy of testing efforts. In the avionics and other critical systems domains, test suites satisfying structural coverage criteria are mandated by standa...
展开
A number of structural coverage criteria have been proposed to measure the adequacy of testing efforts. In the avionics and other critical systems domains, test suites satisfying structural coverage criteria are mandated by standards. With the advent of powerful automated test generation tools, it is tempting to simply generate test inputs to satisfy these structural coverage criteria. However, while techniques to produce coverage-providing tests are well established, the effectiveness of such approaches in terms of fault detection ability has not been adequately studied. In this work, we evaluate the effectiveness of test suites generated to satisfy four coverage criteria through counterexample-based test generation and a random generation approach—where tests are randomly generated until coverage is achieved—contrasted against purely random test suites of equal size. Our results yield three key conclusions. First, coverage criteria satisfaction alone can be a poor indication of fault finding effectiveness, with inconsistent results between the seven case examples (and random test suites of equal size often providing similar—or even higher—levels of fault finding). Second, the use of structural coverage as a supplement—rather than a target—for test generation can have a positive impact, with random test suites reduced to a coverage-providing subset detecting up to 13.5 percent more faults than test suites generated specifically to achieve coverage. Finally, Observable MC/DC, a criterion designed to account for program structure and the selection of the test oracle, can—in part—address the failings of traditional structural coverage criteria, allowing for the generation of test suites achieving higher levels of fault detection than random test suites of equal size. These observations point to risks inherent in the increase in test automation in critical systems, and the need for more research in how co- erage criteria, test generation approaches, the test oracle used, and system structure jointly influence test effectiveness.
收起
摘要 :
Service choreographies present numerous engineering challenges, particularly with respect to testing activities, that traditional design-time approaches cannot properly address. A proposed online testing solution offers a powerful...
展开
Service choreographies present numerous engineering challenges, particularly with respect to testing activities, that traditional design-time approaches cannot properly address. A proposed online testing solution offers a powerful, extensible framework to effectively assess service compositions, leading to a more trustworthy and reliable service ecosystem. The first Web extra at http://youtu.be/JseoxkY5H60 is a video of ServicePot, developed by Unicam-CNR, an extensible registry for choreography-based solutions. The second Web extra at http://youtu.be/3-3kpG3gI3c is a video demonstration of the ParTes (participant testing) tool that shows how it generates skeletons of test cases from a BPMN2 choreography specification.
收起
摘要 :
Context: Software development organizations need testers with high skill levels in a broad range of technical areas and application domains. Accordingly, we need a better understanding of how testers meet such skill demands in the...
展开
Context: Software development organizations need testers with high skill levels in a broad range of technical areas and application domains. Accordingly, we need a better understanding of how testers meet such skill demands in the practice of their role. Objective: This work aims to deepen the understanding of the typical tester role. Method: We performed a thematic analysis of 19 in-depth, semi-structured interviews with software testers working in various industries. To investigate employers' views on such roles, we conducted a thematic analysis of 400 job ads. Results: From the interviews, we identified five subroles of software testers: domain-specific tester, test automation specialist, test infrastructure specialist, user experience tester, and test manager. Most of the practitioners preferred to develop skills and act in one subrole. In contrast, most of the job ads requested that testers act in multiple subroles. Conclusion: Our findings provide a deeper understanding of the tester role, which may guide testers in their acquisition of skills and employers in the recruiting of testers.
收起
摘要 :
Most of the time, customers request complex business logic to be implemented in software applications. Therefore, as long as business requirements grow, the pressure increases on the testing team to deliver the product with high q...
展开
Most of the time, customers request complex business logic to be implemented in software applications. Therefore, as long as business requirements grow, the pressure increases on the testing team to deliver the product with high quality in a very tight time. Manual testing is not suitable for critical and complex applications in terms of both human resources and time. Therefore, there is a strong need to propose an automated testing framework which could reduce the overall software testing time. Automation testing has been introduced to overcome manual testing problems. This study aims to propose a new automated testing framework for testing web applications that enhances the automating process. The proposed framework can save approximately about 75% of the total time/effort involved in the automation process using traditional automation and 21% compared to using Selenium IDE.
收起
摘要 :
Many large-scale software systems must service thousands or millions of concurrent requests. These systems must be load tested to ensure that they can function correctly under load (i.e., the rate of the incoming requests). In thi...
展开
Many large-scale software systems must service thousands or millions of concurrent requests. These systems must be load tested to ensure that they can function correctly under load (i.e., the rate of the incoming requests). In this paper, we survey the state of load testing research and practice. We compare and contrast current techniques that are used in the three phases of a load test: (1) designing a proper load, (2) executing a load test, and (3) analyzing the results of a load test. This survey will be useful for load testing practitioners and software engineering researchers with interest in the load testing of large-scale software systems.
收起
摘要 :
Testing is a destructive task in which the goal is to find relevant defects as early as possible. It requires automation to reduce cost and ensure high regression, thus delivering determined quality. This article reviews technolog...
展开
Testing is a destructive task in which the goal is to find relevant defects as early as possible. It requires automation to reduce cost and ensure high regression, thus delivering determined quality. This article reviews technologies for test automation.
收起
摘要 :
With the increasing importance, size, and complexity of automated test suites, the need exists for suitable methods and tools to develop, assess the quality of, and maintain test code (scripts) in parallel with regular production ...
展开
With the increasing importance, size, and complexity of automated test suites, the need exists for suitable methods and tools to develop, assess the quality of, and maintain test code (scripts) in parallel with regular production (application) code. A recent review paper called this subarea of software testing software test code engineering (STCE). This article summarizes STCE tools, techniques, and guidelines. It also presents specific quantitative examples in this area based on experience in projects and raises important issues practitioners and researchers must address to further advance this field.
收起
摘要 :
Context: Current software-development tools that are used in practice make understanding the test execution of software difficult, for both granular tasks (e.g., answering questions such as, "which test cases execute this method?"...
展开
Context: Current software-development tools that are used in practice make understanding the test execution of software difficult, for both granular tasks (e.g., answering questions such as, "which test cases execute this method?") and global tasks (e.g., answering questions such as, "what is the proportion of unit tests to system tests?"). Current tools typically support local, file-based views of a project's test suite and its execution data, and rarely offer a global overview. Even more rarely, do they provide access to historical information of this nature. Such global overviews can provide a larger context for a method's execution by test cases; help identify other similar, or related methods; and even reveal similarity between individual tests. Objective: This work approaches such challenges with a novel, interactive, matrix-based visual interface that provides a global overview of a software project's test suite, specifically in the context of the methods available in the project's codebase. Through a series of interactive functions to sort, filter, query, and explore a testmatrix visualization, we evaluate how developers can effectively answer questions about their project's test suite, and the code executed by such tests.Method: We built a dynamic test-suite analysis and software-visualization tool that implements our designed interface to address the challenges of understanding the testing of software systems. With this implementation, we conducted a user study of 20 software developers to assess their ability to understand and report test execution information and measured accuracy and time. Additionally, we present a series of case studies to demonstrate a number of insights that our tool reveals.Results: Our evaluations, performed on 26 real-world software systems, show that the interactive visualization assisted developers to answer questions about software tests and the code they execute. Further, the visualization consistently outperforms traditional development tools, both in accuracy and time taken to complete software-engineering tasks.Conclusion: Global-overview test matrices offer novel perspectives on test-suite composition, which can guide software development and testing practices.
收起
摘要 :
Programming Testing is a procedure of discovering mistakes while executing a program with the goal that we get a zero imperfection programming. It is gone for assessing the capacity or convenience of a program. Programming testing...
展开
Programming Testing is a procedure of discovering mistakes while executing a program with the goal that we get a zero imperfection programming. It is gone for assessing the capacity or convenience of a program. Programming testing is a critical method for getting to nature of programming. In spite of the fact that a great deal of headways have been done in formal strategies and confirmation procedures, still we require programming to be completely tried before it could be taken care of to the client side. In this way there are various trying strategies and apparatuses made to achieve the assignment. Programming testing is a critical territory of research and a great deal of improvement has been made in this field. In this paper, testing systems and instruments have been depicted. Some ordinary most recent explores have been abridged. Programming testing is increasing increasingly significance later on.
收起
摘要 :
Today, the growing size and complexity of software along with decreasing development times causes tremendous challenges to software testing. This has driven the whole software industry to seek new ways to test more efficiently and...
展开
Today, the growing size and complexity of software along with decreasing development times causes tremendous challenges to software testing. This has driven the whole software industry to seek new ways to test more efficiently and effectively. Software reuse has been practiced for decades and successful industrial studies have demonstrated such profits as increased productivity and quality as well as decreased development times and costs. This raises the question of whether software reuse could be applied to a testing context as well. This work studies the reuse of tests that are created with a new test specification and implementation language TTCN-3 (Testing and Test Control Notation). In order to apply reuse into a testing context, a set of guidelines for reusable TTCN-3 code is presented. These guidelines are based on the techniques familiar from software reuse, TTCN-3 test system and language characteristics, and on some of the specifics of software testing. Applicability of the guidelines, and the level and profits of TTCN-3 test reuse are determined in a case study. The case study plainly demonstrates that the majority of the guidelines were successfully applied and that they had a positive impact on measured levels and profits of reuse. The overall results, experiences and impressions of TTCN-3 test reuse during this work were very encouraging and will hopefully lead to future projects in areas of test reuse.
收起
原文获取